Models

Bayesian Structure Time Series

BSTS Model is a Structural Time Series Model.

Observation Equation: \[y_t = Z_t^T \alpha_t + \epsilon_t \ \ldots \ \epsilon_t \sim N(0, H_t)\] Transition Equation: \[\alpha_t = T_t \alpha_t + R_t \eta_t \ \ldots \ \eta_t \sim N(0, Q_t)\] \(Z_t\), \(T_t\) and \(R_t\)are model matrices - having a mix of known values and unknown estimable parameters.

The basic structural model we make use of is as follows

\[ log(y_t) = \mu_t + \tau_t + \beta^T x + \epsilon_t\] \[ \mu_t = \mu_{t-1} + \delta_{t-1} + \nu_t \] \[ \delta_t = D + \phi (\delta_{t-1} - D) + \eta_t \] \[ \tau_t = \sum_{s=1}^{P} \tau_{t-s} + \omega_t\] Current level of the trend is \(\mu_t\), curent slope is \(\delta_t\). The seasonal component is \(\tau_t\), where P is the number of seasons in a year. P is strictly integral here. The semi-local linear trend model is similar to the local linear trend, but more useful for longterm forecasting. It assumes the level component moves according to a random walk, but the slope component moves according to an \(AR_1\) process centered on a potentially nonzero value D.

Trigonometric Box-Cox transform, ARMA errors, Trend, and Seasonal components model

TBATS is a State space time series model.

\[y_t^{(\omega)} = Box- Cox (y_t, \omega)\] \[ y_t^{(\omega)} = l_{t-1} + \phi \ b_{t-1} + s_t + d_t\] \[ l_t = l_{t-1} + \phi \delta + \alpha \ d_t\] \[ b_t = (1-\phi) B + \phi \ b_t + \beta \ d_t \] Seasonal component involves a trigonometric representation using Fourier series.

\[ d_t = \sum_{i=1}^{p} \phi_i d_{t-i} + \sum_{i=i}^{q} \theta_i \epsilon_{t-i} + \epsilon_t \]

Here, \(l_t\) is the local level, B is the long run trend, \(b_t\) is the local slope. \(d_t\) denotes an \(ARMA\ (p,q)\) process. \(\epsilon_t\) is white noise. The others are tuning parameters.

Phase 1

Plotting Seasonality

We will do an analysis of the current forecasting model for each quarter.

	Training Data Ends	Test Data Starts	Test Data Ends	Number of Stores
Quarter 1	11748	11801	11813	1026
Quarter 2	11709	11714	11726	585
Quarter 3	11722	11727	11739	536
Quarter 4	11735	11740	11713	828

Quarter 1

Time period: 11801 - 11813
Training period: Upto 11748.
1026 stores studied.
This quarter has 1 major festival: Easter, the 10th week of the quarter.
Current Model: BSTS + Walmart. Current accuracy stands at: 13%
Proposed Shift to: BSTS + Custom. Improved accuracy would be: 11%
Easter festival is a problem zone. We will come back to this later.

Quarter 2

Time period: 11714 - 11726.
Training period: Upto 11709.
585 stores studied.
This quarter has no major festival.
Current Model: BSTS + Walmart. Current accuracy stands at: 19%
Proposed Shift to: TBATS + Custom. Improved accuracy would be: 15%

Quarter 3

Time period: 11727 - 11739.
Training period: Upto 11722.
536 stores studied.
This quarter has no major festival, but sales were greatly affected by hurricanes. The most affected week was the 7th week, because of Hurricane Harvey.
Current Model: BSTS + Walmart. Current accuracy stands at: 13.2%
Proposed Shift to: BSTS + Custom. Improved accuracy would be: 12.7%

Quarter 4

Time period: 11740 - 11752.
Training period: Upto 11735.
828 stores studied.
This quarter has 2 major festivals: Thanksgiving and Christmas.
Current Model: BSTS + Walmart. Current accuracy stands at: 25%
Proposed Shift to: BSTS + Custom. Improved accuracy would be: 19.2%

Phase 2

Introduction.

Focus on three time points: Easter, Thanksgiving and Christmas.
We want to find those stores due to which the models are experiencing an unexpected spike in forecasting error.
We would ideally want a damped linear-like trend of weekly MAPE values for each quarter. Based on this expectation, we formulate some cutoff of the MAPE values. If value is less, then store is GOOD, else it is BAD.
This is done for all three festivals.
Cutoff values:
- For Easter: 0.32
- For Thanksgiving: 0.33
- For Christmas: 0.82.

Finding driver variables.

Suppose we already know (for a training set) if a store is GOOD/BAD.
Each of these stores has some demographic/geographic behaviour.
We want to see if we can leverage this information to build a classification model.
We use a technique called ANOVA to find how useful a variable will be in classifying a store.
ANOVA is used to analyze the differences among group means in a variable that can be classified into two or more classes.
In the ANOVA framework, the observed variance in a particular variable is partitioned into components attributable to different sources of variation - here, that component is the classification of a store as GOOD or BAD.
Also, if the group means are close to each other, then the variable is not a good going to be useful in classification. This is exploited by the ANOVA model.

Classification

Variable Importance

Confusion Matrices

L: Easter, C: Thanksgiving, R: Christmas

Phase 3:

We can use the driver variables identified in Phase 2 to build a regression model:

\[Scaling\ Factor = f(x_1, x_2, \ldots,x_p)\]

\(x_1, \ldots, x_p\) are the identified driver variables. Multicollinearity is a big problem in this approach, so we abandon this method.

Instead, we repeat an exercise in classification.
- find a common scaling factor for all Bad stores for Easter, Thanksgiving and Christmas.
- We use median of MAPE values of Bad stores. Obviously, this figure will not be available for the next cycle.
- We can either use these figures next year as well, or find a model which helps in finding this scaling factor as a function of the demographic information of the Bad stores. (Under exploration.)
- \(SF_E = 0.68\), \(SF_T = 0.66\) and \(SF_C = 0.48\)

Phase 4.

Introduction

We will judge the efficacy of implementing Phase 4.

Improvement in overall performance due to scaling:
- Quarter 1: from 11.25%, MAPE drops to 11.09% [16 basis points]
- Quarter 4: from 20.40%, MAPE drops to 19.90% [50 basis points]
Improvement in performance of forecasts of Bad stores due to scaling:
- Easter: 51.2%% to 29.2%
- Thanksgiving: 52% to 35%.
- Christmas: 110% to 58%.

Forecasting of OG orders

Appendix